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LINEAR  TRANSFORMATIONS,  PROJECTION  OPERATORS  AND 
GENERALIZED  INVERSES-A  GEOMETRIC  APPROACH 

ABSTRACT  ,  f- 

A  generalized  inverse  of  a  linear  transformation  A:  v  -» w,  where  v  and 
w  are  finite  dimensional  vector  spaces,  is  defined  using  geometric  concepts  of 
linear  transformations  and  projection  operators.  The  inverse  is  uniquely 
defined  in  terms  of  specified  subspaces  m  Cv,  l  Cw  and  a  linear 

transformation  N  such  that  AN  =  O,  NA  =  O.  Such  an  inverse  which  is  unique  is 

called  the  ImN -inverse.  A  Moore-Penrose  type  inverse  is  obtained  by  putting 
N=0.  Applications  to  optimization  problems  when  v  and  w  are  inner  product 

spaces,  such  as  least  squares  in  a  general  setting,  are  discussed.  The  results 

given  in  the  paper  can  be  extended  without  any  major  modification  of  proofs 
to  bounded  linear  operators  with  closed  range  on  Hilbert  spaces. 


Key  Words  and  Phrases:  g-inverse,  Linear  transformation,  Moore-Penrose 

inverse.  Projection  operator. 

AMS  classification  index:  15A04,  15A09 
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1.  INTRODUCTION 


Let  v  and  w  be  two  finite  dimensional  vector  spaces  and  A:v  -*  w  be  a 
linear  transformation  from  v  to  w  .  We  denote  the  range  of  A  by 


R(A)  =  {y:  Ax=y,x  e  v  }  =  a  Cw 


(1.1) 


and  the  nullity  or  kernel  of  A  by 

K(A)  =  {x  e  v  :  Ax  =  0}  =  k  C  v  .  (1.2) 

If  c  and  d  are  subspaces  of  v  such  that 

end  =  0,  the  null  vector  (1.3) 

C®d  =  {x1  +  x2:x1ec.x2ed}  =V  (1.4) 

then  c  and  d  are  said  to  be  direct  complements.  Given  any  subspace  c  C  v, 
there  exists  a  subspace  d  C  v  ,  called  the  direct  complement  of  c,  which 
satisfies  the  conditions  (1.3)  and  (1.4).  The  choice  of  d  is  not  unique.  Given 
any  pair  c,  d  satisfying  (1.3)  and  (1.4),  any  vector  x  €  v  has  the  unique 
decomposition 


x  =  x1  +  x2,  Xjfic,  x2ed.  (1.5) 

The  transformation  x  -»  Xj,  which  is  linear,  is  called  the  projection  operator  on 
c  along  d  and  is  represented  by 


R(pc j)  = c-  R(pd.c)  =  * 


where  I  and  O  are  identity  and  null  transformations  on  v. 

A:  v  ->  w  is  called  bijective  if  it  is  onto  and  one-to-one.  In  such  a  case, 
there  is  an  inverse  of  transformation  A,  denoted  by  A'1,  which  is  linear  and 


is  such  that 


A  A*1  =  I(on  w).  A"1  A  =  I(on  v). 


The  restriction  of  A:  v  ->  w  to  a  subspace  c  C  v  is  also  a  linear 
transformation  and  is  denoted  by 

Ale:  c  w  .  (1.10) 


Let  k  be  the  nullity  of  A  as  defined  in  (1.2),  m  be  any  direct  complement  of  k, 
(i.e.  kQm  =  v),  and  a  be  the  range  of  A.  Then 


Aim  :  m  -»  a  is  a  bijection. 


(1.11) 


This  is  easily  seen  as  follows.  Choose  yea  and  let  x  6  v  be  a  preimage  of  y,  i.e. 
x  is  such  that  Ax  =  y.  Then 


P  ,  x  e  m  and  Ax  =  A(P  ,  +  P,  x)  =  AP  ,  x  =  y 
“  *  m.k  k.m  ~  *•  3 


so  that  the  mapping  (1.11)  is  onto.  Further  if  Xj  e  m  and  e  m,  then  Xj  -  x^  e 


m.  If  now  AXj  =  Ax^,  then  A(Xj-x2)  =  0  =>  Xj  -  x^  €  k  which  is  a  contradiction 


unless  Xj  =  x^.  Thus  Xj  #x2=»  Ax^Ax^  which  shows  that  the  mapping  (1.11)  is 


one  to  one. 


The  result  (1.11)  implies  the  existence  of  an  inverse 


(Aim)’  =  T  :  a  -»  m 
tn 


(1.12) 


such  that 


T  Alm  =  I(onm),  AT  A  =  A,  T  A  =  P  .  (1.13) 

m  mm  m.k 


AT  I  a  =  I(on  a),  T  AT  =  T  . 
m  m  m  m 


(1-14) 


iTOWl 


uwmm  u  t;  to  w  to  w  wv.v.-wv 


> 
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Note  that  T  is  defined  only  on  a  C  w  and  not  the  whole  of  w. 

171 

In  this  paper,  we  develop  the  concept  of  a  generalized  inverse  (g- 

inverse)  of  A  when  it  is  not  necessarily  bijective  and  provide  some 
characterizations  and  applications.  This  is  accomplished  by  extending  Tm  as  a 


t 


linear  transformation  on  any  complement  of  a  in  w . 


i 


2.  G-INVERSE  OF  A  LINEAR  TRANSFORMATION 


Let  A:  v  -»  w  be  a  linear  transformation  with  range  a  C  w  and  nullity 
i  C  v.  Consider  a  consistent  linear  equation  Ax  =  y  (e  a  )  and  denote  the  set  of 
solutions  by 


Sy  =  {  x  6  v  :  Ax=  y).  (2.1) 

Suppose  that  there  exists  a  linear  transformation  G:  w  -»  v  such  that 

Gy  €  Sy  for  any  y  €  a  (2.2) 

i.e.,  Gy  is  a  solution  of  Ax  =  y  for  any  yea.  Then  substituting  Gy  for  x  in  Ax  =  y 

AGy  =  y  V  y  e  a  or  AGIa  =  I(on  a )  =»  AGA  =  A.  (2.3) 

Conversely,  if  there  exists  a  G  such  that  (2.3)  holds  and  Ax  =  y  is  a  consistent 
equation,  i.e.,  there  exists  an  x  such  that  Ax  =  y,  then 

AGA  =  A  =*  AGAx  =  Ax  or  AGy  =  y  (2.4) 

so  that  Gy  is  a  solution  of  Ax  =  y.  Any  G  satisfying  the  condition  (2.3)  is  called  a 
g-inverse  of  A;  this  enables  us  to  solve  a  consistent  equation  Ax  =  y.  Such  a  G 

which  may  not  be  unique  is  denoted  by  A'  and  the  set  of  all  G  satisfying  (2.3)  is 

denoted  by  {A'}.  The  following  lemmas  characterize  g-inverses. 

Lemma  2.1  Let  k  be  the  nullity  of  A,  a  the  range  of  A,  and  G  be  a  linear 
transformation  such  that  AGA  =  A,  i.e.,  G  e  (A'}.  Then  the  following  hold: 

(i)  m  =  R(GA)  is  a  direct  complement  of  k,  and  :  a  -»  m  =  Gin  or 

GAIm  =  I(on  m),  where  T  =  (Aim  )  1 . 

m 

(ii)  If  N  =  G  -  GAG,  then  AN  =  0  and  NA  =  O. 

(iii)  /  =  R(I-AG)  is  a  direct  complement  of  a,  and  Nl/  =  Gl /. 


Proof  of  (i).  Let  x  e  m  G  k  .  Then  x  =  GAu  for  some  u  and  Ax  =  AGAu  = 
Au  =  0  =>  x  =  0,  i.e.,  m  n  k  =  0.  Also,  if  x  e  v  ,  then 

x  =  GAx  +  (I-GA)x  =  xj  +  x2 

where  x^  6  m  and  x^  e  k  since  Ax^  =  0.  Thus  the  conditions  (1.3)  and  (1.4)  are 
satisfied  so  that  m  is  a  direct  complement  of  k  .  Then  there  exists  T^  as 
defined  in  (1.12)  -  (1.14),  and  in  particular  T^AIm  =  I.  Now 


T  Ax  =  T  AGAx  =  T  A  (GAx)  =  GAx 
mm  m 


since  GAx  e  m  ,  which  implies  T  =  Gla  or  GAIm  =  I. 

Tfl 


Proof  of  (in  .  Obvious. 

Proof  of  (iii).  Let  y  el  Ha.  Then,  for  some  x  and  u 

y  =  Ax  =  (I-AG)u  =*  AGAx  =  AG(I-AG)u  =  0  =*  Ax  =  0 
so  that  /  ft  a  =0.  Further 

y  =  AGy  +  (I-AG)y  =  y  =  +  y2 

where  y(  ea  and  y 2  €  /  .  Thus  the  conditions  (1.3)  and  (1.4)  are  satisfied  so 
that  /  is  a  direct  complement  of  a.  Consider 

N(I-AG)u  =  Nu  =  (G-GAG)u  =  G(I-AG)u 
for  any  u,  so  that  Nl/  =  G\l  .  Lemma  1  is  proved 

Definition.  Let  m  be  any  direct  complement  of  k  (the  nullity  of  A)  and 
/  be  any  direct  complement  of  a  (the  range  of  A)  and  N:w  -»  v  be  any  linear 
transformation  such  that  AN  =  O  and  NA  =  0.  Then  G  is  said  to  be  an  Im N- 
inverse  of  A  if 

(i)  G  is  linear,  (ii)  GAIm  =1  and  (iii)  Gl /  =  Nl/ .  (2,5) 


pa  | 

[jki 

Si 


I.L'Ul.llU'UU'i. 


V  ■  V  -l  *-  V  *.  v 


»  ’  v"  *.’  ^  ■  r  *< 


Such  a  G,  if  it  exists,  is  denoted  by  G^N-  Lemma  2.2  characterizes  such 


inverses. 


Lemma  2.2. 


G.  =T  P  ,  +  N 
imN  m  al 


where  is  as  defined  in  (1.12)  with  respect  to  a  chosen  m,  so  that  G^N  is 


unique  for  any  given  l  ,  m  and  N . 


Proof.  Let  y  =  yj  +  y2>  y2  e  a  and  y2  e  /  .  Then 


G/mN  y  ~  C/mNyi  +  C/mN  y2 


=  Tffl  y}  +  Nyr  using  (2.5) 


=  T  P  .  y  +  Ny 
m  ad  3  3 


=  (T  P  ,  +  N)y  V  y  €  w 
m  ad  3  3 


which  establishes  Lemma  2. 


Lemma  2.3.  G/mN  €  (A'}  and  the  mapping 


(/ ,  m  ,  N)  -»  (A"}  is  bijective. 


Proof.  Using  the  representation  (2.6) 


AG,  ..  A  =  A(T  P  ,  +  N)A 
ImN  m  a  l 


=  AT  P  ,  A  =  AT  A  =  A,  using  (1.13) 
m  ad  m 


1 


s0  11131  G!mN  €  {A’i- 


Again  the  representation  (2.6)  shews  that  N  is  unique  for  given  /  , 

m,  N.  On  the  other  hand,  it  is  shown  in  Lemma  2.1  that  given  a  G  e  {A’}  ,  there 
exists  an  l  ,  m,  N  for  which  it  is  the  ZmN  -inverse,  and  the  result  (2.7)  is 
established. 

Incidentially,  the  existence  of  a  g-inverse  G  satisfying  the  condition 
AGA  =  A  is  established  through  the  representation  (2.6). 

The  anatomy  of  a  g-inverse  is  exhibited  in  the  following  diagram. 


A  :  V  ->  W 


T  .  a  ->  m 

m* 


N  :a  ->0 


N  :  l  ->  k 


G  :  w  ->  V 


Lemma  2.4  .  Let  k,  m,  a,  l  and  N  be  as  given  in  the  definition  above  the 
equation  (2.5).  Then  the  following  statements  on  a  linear  transformation  G  are 
equivalent. 


P,  G  =  N. 
k.m 

G  -  GAG  =  N. 


(i) 

GAIm  =  I, 

Gl /  =  NIZ  ( 

(ii) 

GA  =  ^  ' 

GP.  =  N. 
l.a 

(iii) 

GA  =  ^m.k’ 

> 

O 

ii 

a*0 

(iv) 

GA  =  Pm.k * 

AG  =  P  ,, 
a.l 

(V) 

(Vi) 
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AGA  =  A  ,  R(Gla  )  =  m,  GP/  =  N. 

AGA  =  A,  R(GIA)  =  m,  R(GI0C*,  G  -  GAG  =  N. 

Proof,  (i)  =*  (ii). 

GAx  =  x  if  x  6  m.  using  the  condition  GAIm  =  I, 
GAx  =  0  if  x  ek,  using  the  condition  Ax  =  0 


which  shows  that  GA  =  P  , .  Further 

m.K 


O' 1  =  N,/  -  GPl.a  =  NPl.a  =  ™La  +  m a.l  =  N' 


(ii)  =>  (iii). 


GP,  =  N  =»  AGP,  =0,  and  A  =  AP  .  =  AGA 
l.a  l.a  m.K 


which  together  imply  AG  =  Pfl  j.  Further 


N  =  GP/fl  =  G(I-Pfl  ^  )  =  G(I-AG) 


-  <'-GA)G  -  =  P*.m  °- 


(iii)  =*  (iv),  since 


P.  =  I  -  GA  and  P,  G  =  G  -  GAG. 
k.m  k.m 


(iv)  =>  (v),  since 


GA  =  P  ,  =*  AGA  =  AP  ,  =  A, 
m.K  m.K 


GA  =  Pm.i  =R(GA)  =  R(Pm,t)  =  m 


GP,  =  G  -  GP  ,  =  G  -  GAG  =  N. 
l.a  a.l 


(v)  =»  (vi),  since 


3.  REFELXIVE  G-INVERSE 

Definition.  G  is  said  to  be  a  reflexive  (or  an  outer  inverse)  of  A  if 

AGA  =  A  and  G  =  GAG.  (3.1) 

The  class  of  G  satisfying  (3.1)  is  represented  by  {A^}  and  any  member  of  it  by 
A'. 

The  following  lemma  characterizes  reflexive  g-inverses 
Lemma  3.1 

{Gfcn0}  =  ^  C  (3  2) 

i.e.,  all  reflexive  g-inverses  are  generated  by  choosing  N  =  O,  and  {/  ,  m)  as  the 
set  of  all  direct  complements  of  a  and  k  respectively. 

Proof.  From  the  representation  (2.6) 


G,  =  T  P  . 
ImO  m  a.l 


Then 


AT  P  ,  A  =  AT  A  =  A,  using  (1.13) 
m  a.l  m 


T  P  .AT  P  .  =  T  AT  P.=T  P,,  using  (1.14) 
m  a  l  m  al  m  m  a.l  m  a.l 


which  shows  that  G^q  €  {A^}. 

Conversely  let  AGA  =  A  and  G  =  GAG.  Choose  m 
Then  as  in  (i)  of  Lemma  2.1 

Gla  =  T  . 
m 

Further,  if  y  el,  then  y  =  (I-AG)u  for  some  u  so  that 


=  R(GA)  and  /  =  R(I-AG). 


Kji 

& 

i 


1 


w.  .-N 


i: 


iVit 


Vit, 


.  .mwy.i'uw 
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Gy  =  G(I-AG)u  =  0,  since  G  =  GAG 
=>  GIZ  =  0. 


Thus  G  satisfies  the  definition  of  GlmO  and  Lemma  3.1  is  proved. 
Lemma  3.2 


G/mN  "  GlmO  +  N 


(3.3) 


Proof.  The  result  (3.3)  follows  from  the  representation  (2.6). 


Lemma  3.3  .  The  following  statements  are  equivalent. 


(i)  G  is  the  /mO-inverse 

(ii)  GA  =  P  .  GIZ  =0 

m.K, 

(iii)  GA  ~  pm  ^  ■  AG  =  ?a  /  >  G  =  GAG. 

(iv)  GA  =  Pm  k  ,  g  =  R(G)  =  m  ,  AG  =  Pfl>/ . 

(v)  AGA  =  A,  g  =  R(G)  =  m  ,  R(GI/ )  =  0 

The  results  are  proved  on  the  same  lines  as  in  Lemma  2.4. 

If  v  and  w  are  inner  product  spaces  and  if  we  choose  m  as  the 
orthogonal  complement  of  k  ,  and  /  as  that  of  a  ,  then  the  condition  (iv)  of 
Lemma  3.3  can  be  written  as 

Gk  =  I-P,  =P  ,  AG  =  P  (3.4) 

k  g'  a 

where  g  =  R(G),  P^  is  the  orthogonal  projector  on  k  and  Pfl  is  the  orthogonal 

projector  on  a,  which  is  equivalent  to  the  definition  given  by  Moore  (1920) 
and  Penrose  (1955). 


4.  L  M  -INVERSE 

If  in  the  definition  of  the  Im  N  -inverse,  we  do  not  specify  N  but  only 
require  that  G\l  C  k  ,  then  we  can  write  the  conditions  for  a  g-inverse  in  the 
form 

GAIm  =  I,  AG!/  =  O.  (4.1) 

We  represent  the  solution  of  (4.1)  by  G^,  which  may  not  be  unique  and  call  it 
an  Im- inverse. 

Lemma  4. 1 .  The  following  statements  are  equivalent  for  given  /  and  m  , 
any  direct  complements  of  a  and  k  respectively. 

(i)  G  is  an  Im- inverse. 

(ii)  GA  =  Pm,T  AG  =  Pa.r 

(iii)  AGA  =  A,  R(Gla  )  =  m  ,  R(GI/)  C  k. 

The  proof  of  Lemma  4.1  follows  on  the  same  lines  as  that  of  Lemma  2.4. 
The  definition  given  in  (ii)  of  Lemma  4.1  was  proposed  by  Langehop 

(1967). 
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5.  OTHER  INVERSES 


5.1  M-inverse 


An  m-inverse  is  defined  by  specifying  m  only  as  in  the  following 


lemma. 


Lemma  5.1.  The  following  conditions  are  equivalent  for  an  m-inverse. 


(i)  GAIm  =  I. 
<U>  GA  =  Pm.k- 


(iii)  AGA  =  A,  R(Gltf)  =  m 


We  represent  an  m  -inverse  of  A,  which  may  not  be  unique,  by  A^  to  be 

consistent  with  the  notation  developed  earlier  by  the  author  (Rao,  1967). 

If  v  is  a  vector  space  endowed  with  an  inner  product,  then  we  may 
choose  m  to  be  an  orthogonal  complement  of  k  .  In  such  a  case,  if  Ax  =  y  is  a 
consistent  equation,  then 


min  llxll  =  IIA^ y II 
Ax=y 


so  that  A^y  is  the  minimum  norm  solution  of  Ax  =  y.  Note  that  any  particular 
solution  of  Ax  =  y  is  given  by  A'y  where  A'  is  any  g-inverse  satisfying  the 


condition  A  A*  A  =  A. 


To  prove  5.1,  we  observe  that  if  x  is  any  solution  of  Ax  =  y,  then 


Afx-A^y)  =  0  =>  x  -  A^y  €  k 


and  by  definition  A'  y  e  m  ,  and  since  m  and  k  are  orthogonal  complements 


llxll2  =  llx-Aj^y  +  A^yll2  =  llx-A^yll2  +  lA^yll2,  by  Pythogorous  theorem 
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5.2  /-inverse 


An  /-inverse  of  A,  denoted  by  Aj  is  a  linear  transformation  G  as 
characterized  in  Lemma  5.2. 


Lemma  5.2.  The  following  conditions  are  equivalent  for  an  /-inverse. 


(i)  AG  =  Pa/. 

(ii)  AGA  =  A,  R(GI/ )  C  k  . 

If  w  is  an  inner  product  space,  we  may  choose  /  to  be  the  orthogonal 
complement  of  a  .  In  such  a  case 

minlly  -  Axil  =  lly  -  A  A.'  yll  (5.2) 

x  1 

so  that  Ajy  is  a  general  least  squares  solution  of  a  possibly  inconsistent 
equation.  Observe  that 


y  -  A  A'  y  =  (l-Pa  l  )y  =  P/>fl  y  e  / 


S3 

fc 

fljjL 

f  -% 

3 

#3 
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A  A7  y  =  P  .yea=>AA7y-Axea 
/  a./  / 

so  that  by  virtue  of  orthogonality  of  /  and  a 

lly- Axil2  =  lly  -  A  Aj  y  +  A  Ajy  -  Axil2 

=  lly  -  A  AJyil2  +  IIA  Ajy  -  Axil2  >  lly  -  A  Aj  y  II2. 


5.3  /mO- inverse 

The  /mO-Inverse,  specified  by  l  ,  m  and  N  =  O  and  demoted  by  Ajm0,  is 
already  characterized  in  Lemma  5.3.  When  v  and  w  are  inner  product  spaces, 
we  may  choose  m  to  be  the  orthogonal  complement  of  k,  and  /  to  be  the 
orthogonal  complement  of  a.  In  such  a  case  the  /mO-inverse  can  be 
characterized  by 


>\\, 

k 

v\V 
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GA  =  (1  -  P,  )  =  P  ,  AG  =  P 

k  g  a 


where  P^  denotes  the  orthogonal  projection  operator  on  x.  The  inverse 
satisfying  the  conditions  (5.3)  is  the  Moore-Penrose  inverse  and  is 
represented  by  A+.  We  show  that 

min{llx.  II  :  lly  -  Ax.  II  =  min  lly  -  Axil)  =  IIA+yll  (5.4) 

V  *  »  ¥ 


where  the  norms  associated  with  x  e  v  and  yew  may  be  different,  so  that  A+y 
is  the  minimum  norm  least  squares  solution  of  Ax  =  y  (possibly  inconsistent 
equation). 

Observe  that 

lly  -  Axil2  =  IIPfl  y  -  Axil2  +  IIP.  y  II2  (5.5) 

where  the  second  member  does  not  involve  x,  and  the  minimum  of  Ily-Axll  is 

attained  if  Ax  =  P  y  so  that  x,  is  a  solution  of  Ax  =  P  y  which  is  a  consistent 
a  l  a 

equation.  The  problem  (5.4)  demands  the  solution  of  Ax  =  Pfl  y  with  the 
minimum  norm,  which  is  supplied  by  the  equation  (5.1) 

x=AmpAy  =  Gy- 


GA  =  A  ‘  P  A  =  A  '  A  =  P  .  =  (I-P,  ) 
m  a  m  m.k  v  k  ' 


AG  =  A  A  *  P  =  P  ,  since  R(P  )  =  a 
m  a  a  a 


so  that  G  satisfies  the  conditions  (5.3)  and  hence  it  is  the  ImO -inverse. 

Thus  the  /  and  /mO -inverses  provide  solutions  to  the  least  squares 
problem  in  the  most  general  setting.  When  v  and  w  are  Euclidian  spaces  of  m 
and  n  dimensions  respectively,  the  linear  transformation  A  can  be 


represented  as  an  nxm  matrix  in  which  case  explicit  representations  can  be 
obtained  for  various  generalized  inverses  as  shown  in  Rao  (1955,  1962,  1967, 
1973)  and  Rao  and  Mitra  (1971).  The  geometric  approach  in  the  case  of  a 
general  linear  transformation  was  developed  in  Rao  and  Yanoi  (1985).  All  the 
results  of  this  paper  can  be  extended  without  any  major  modification  of  the 
proofs  to  bounded  linear  operations  with  closed  range  on  Hibert  spaces. 
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